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TOWARDS  JUSTIFYING  ENLISTMENT  STANDARDS: 
LINKING  INPUT  CHARACTERISTICS  TO  JOB  PERFORMANCE 


-‘The  Job  Performance  Measurement  (JPM)  Project  has  broken  new 
ground  by  embarking  on  the  large-scale  development  and  administration 
of  hands-on  job  performance  tests.  In  the  next  stage,  this  pioneering  effort 
can  go  even  further  in  helping  to  rationalize  the  selection  and  classification 
process  by  evaluating  the  cost-effectiveness  of  enlistment  standards.  The 
Department  of  Defense  has  never  had  a  sound  empirical  basis,  in  terms  of 
job  performance,  for  setting  minimum  standards  or  for  establishing  the  dis¬ 
tribution  of  aptitudes  and  educational  levels  required  of  enlisted  recruits. 
If  the  JPM  Project  is  carried  through  to  a  successful  conclusion,  the  joint 
efforts  of  the  services  can  provide  such  an  empirical  basis. 

‘The  process  of  evaluating  the  cost-effectiveness  of  enlistment  standards, 
which  includes  determining  the  mix  of  aptitude  levels  required  to  sustain 
the  career  enlisted  force,  may  be  characterized  as  follows: 

rV  Measure  job  performance^ 

•  Establish  the  relationship  between  job  performance  and  accession  at¬ 
tributes  (e.g.,  aptitude,  education,  job  experience,  etc.); 

>  -•  Determine  constraints  on  the  linkage  model, 

-*  Estimate  the  costs  associated  with  operating  the  force.  *  ■  * 

-  •  Evaluate  alternative  enlistment  standards  for  their  cost-effectiveness. 

Each  of  these  steps  are  examined  and  discussed  in  light  of  their  impact  on 
setting  justifiable  enlistment  standards.1 

‘For  this  paper,  “standard”  is  used  in  the  context  of  determining  an  enlistment  standard 
or  cutoff  score  on  an  aptitude  composite  used  for  entrance  into  a  particular  military 
occupational  specialty.  This  is  in  contrast  to  the  other  papers  of  this  symposium  that 
use  the  term  to  refer  to  determining  score  intervals  on  the  criterion  performance  measure 
for  categorization  of  job  competencies. 


Measuring  Job  Performance 

For  the  purpose  of  linking  job  performance  to  enlistment  standards, 
the  performance  measure  must  be  an  accurate  reflection  of  an  individual’s 
ability  to  perform  his  or  her  job  responsibilities.  For  this  to  be  true,  the 
performance  test  must  sample  job  tasks  from  the  entire  spectrum  of  job 
requirements,  not  just  from  those  tasks  that  are  more  difficult,  complex,  or 
most  frequently  performed. 

If  a  test  is  designed  so  that  it  is  more  difficult  than  the  true  job,  then  the 
enlistment  standard  resulting  from  this  harder  test  would  be  higher  than  the 
enlistment  standard  generated  by  a  test  of  true  job  tasks.  That  is,  given  a 
moderate  correlation  between  performance  and  aptitude,  more  lower-ability 
personnel  will  be  eligible  based  on  the  representative  test  as  opposed  to  the 
harder  test.  Thus,  while  lower-ability  individuals  can  satisfactorily  perform 
the  full  range  of  job  responsibilities,  they  can  not  necessarily  perform  the 
most  difficult  job  tasks.  Therefore,  without  explicit  justification  for  the 
sampling  of  test  content  from  the  full  domain  of  job  requirements,  the  link¬ 
age  of  test  performance  and  enlistment  standards  could  result  in  standards 
that  are  too  high  or  low,  depending  on  the  direction  of  bias  in  task  sampling. 

If  the  performance  test  is  properly  constructed  so  that  the  full  range 
of  job  requirements  are  represented  and  the  test  score  reflects  how  much 
of  the  job  an  individual  can  perform,  then  the  score  resulting  from  this 
test  is  a  competency  score.  That  is,  a  score  of  75  percent-correct  on  the 
performance  measure  implies  that  the  individual  can  perform  75  percent  of 
the  job  requirements.  Whether  the  continuous  percent-correct  score  should 
be  categorized  into  mastery  levels,  either  for  convenience  or  for  other  ap¬ 
plications,  is  not  an  issue  that  directly  relates  to  the  setting  of  enlistment 
standards. 

It  is  recommended  that  the  expenditure  of  experts’  knowledge  be  in¬ 
vested  in  the  definition  of  job  requirements  and  task  sampling  so  that  the 
linkage  of  test  scores  to  the  domain  of  job  requirement  will  result  in  a  con¬ 
tinuous  competency  score  scale.  In  this  manner,  the  distribution  of  test 
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scores  will  mirror  the  distribution  of  proficiency  within  the  specialty.  The 
use  of  experts’  time  and  resources  on  the  difficult  task  of  determining  cutoff 
scores  that  define  broad  qualitative  categories  of  competency  (e.g.,  mastery 
versus  nonmastery)  on  an  unrepresentative  task  score  scale  is  not  produc¬ 
tive  and  misdirected  for  the  purpose  of  establishing  enlistment  standards. 

Finally,  it  is  also  important  to  note  the  policy  ramifications  for  the 
defining  of  job  requirements  and  sampling  of  job  tasks  for  the  test.  These 
decisions  can  not  be  made  by  testing  psychologists  or  researchers,  but  rather 
the  decision  process  must  readily  involve  service  job  experts,  subject  matter 
experts,  and  policy  representatives. 

Relating  Job  Performance  to  Accession  Attributes 

To  directly  link  enlistment  standards  to  job  performance,  it  is  necessary 
to  determine  the  relationship  between  the  attributes  of  accessions  (e.g.,  ap¬ 
titude,  educational  level,  job  experience)  and  the  hands-on  test  scores.  De¬ 
pending  upon  the  input  characteristics  available  for  the  accessions  group, 
the  prediction  of  job  performance  can  be  quite  complex.  For  the  anal¬ 
ysis  of  the  upcoming  Marine  Corps  testing  of  the  Infantry  Occupational 
Field,  four  input  characteristics  have  been  identified:  aptitude,  probability 
of  retention,  time  in  service  or  experience,  and  educational  level.  Each  of 
these  characteristics  have  been  shown  to  strongly  impact  the  prediction  of 
hands-on  tost  performance  [l].  Note  that  the  Armed  Forces  Qualification 
Test  (AFQT)  is  not  included  since  it  is  highly  correlated  with  the  General 
Technical  aptitude  composite  used  for  the  Infantry  specialties. 

Description  of  the  upcoming  Marine  Corps  testing  of  the  Infantry  Occu¬ 
pational  Field  will  help  in  developing  the  regression  of  hands-on  test  perfor¬ 
mance  (called  predicted  performance  throughout  this  paper)  on  the  acces¬ 
sion  attributes.  The  Marine  Corps  will  test  approximately  200  infantrymen 
at  each  of  four  yearly  intervals  during  the  first  term  of  enlistment.  For  all 
of  these  individuals,  aptitude  composite  scores  will  be  available.  The  prob¬ 
ability  of  retention  at  each  of  the  four  time  intervals  and  for  the  completion 
of  training  will  be  known  from  other  data  sources.  Both  high  school  and 


3 


non-high  school  graduates  will  be  included  in  the  sample.  The  relationships 
between  these  variables  is  given  by  the  following  regression: 

5 

Pp  —  Bo  +  A  +  R,  [T,  +  AT,)  -+- 1  :  E.  (1) 

t=i 

where  Pp  =  predicted  hands-on  performance 

Bo  =  grand  mean  or  constant  for  difficulty  of  MOS 
A  =  aptitude  composite  score 
Ri  =  probability  of  retention  at  time  interval  i 
Ti  =  time  in  service  interval  (dummy  variable) 

AT,  =  aptitude-time  interval  interaction 
<  =  error 

E  =  educational  level  (HSG,  NHSG). 

The  above  regression  equation  assumes  that  different  coefficients  will 
be  necessary  to  express  the  relationship  between  hands-on  job  performance 
and  the  input  characteristics  of  aptitude,  retention,  and  time  in  service  for 
high  school  versus  non-high  school  graduates.  That  is,  while  non-graduates 
on  average  have  lower  aptitude,  their  performance  generally  improves  faster 
with  time  in  service  than  graduates  with  higher  aptitude.  Accordingly,  the 
regressions  weights  for  these  variables  will  differ  depending  upon  educa¬ 
tional  status.  Likewise,  there  are  different  retention  rates  throughout  the 
first  term  for  these  two  groups.  The  effects  of  time  in  service  and  the  in¬ 
teraction  of  time  and  aptitude  are  weighted  by  the  probability  of  retention 
through  these  five  time  periods  (completion  of  training  plus  yearly  reten¬ 
tion  rates  for  four  years).  The  time  intervals  are  represented  by  dummy 
variables,  so  in  essence  this  is  five  separate  regressions  because  an  individ¬ 
ual  can  only  be  in  one  time  interval.  The  overall  regression  assumes  that 
retention  and  aptitude  are  not  correlated:  however,  if  they  are,  this  inter¬ 
action  term  can  also  be  included  in  the  equation. 

The  regression  in  equation  1  is  one  possible  formulation  of  the  rela¬ 
tionship  of  job  performance  and  accession  attributes.  Given  other  input 
characteristics  or  even  composite  definitions  of  job  performance,  the  struc¬ 
ture  of  the  regression  will  most  likely  change  significantly.  An  important 


issue  for  the  joint  service  project  to  address  is  the  stability  of  the  regression 
weights  across  the  variety  of  surrogate  measures  that  are  being  developed 
(e.g.,  job  knowledge  tests,  walk-through  interviews,  task  simulations,  and 
ratings).  To  the  extent  that  the  weights  are  not  similar,  the  case  for  sub¬ 
stitutability  for  the  hands-on  tests  is  weakened. 

Determining  Model  Constraints 

In  evaluating  the  cost-effectiveness  of  enlistment  standards,  either  cost 
or  performance  must  be  fixed,  and  then  the  other  factor  allowed  to  vary. 
The  argument  advanced  here  is  that  performance  should  be  held  constant 
and  then  cost  minimized  for  attaining  that  level  of  performance.  In  prac¬ 
tice,  of  course,  cost  is  fixed  to  a  large  degree  by  budgetary  constraints. 
With  adequate  justification,  however,  budgets  can  be  changed.  A  thrust 
of  the  JPM  Project  is  to  provide  a  sound  empirical  basis  for  obtaining  the 
funds  required  to  procure  and  maintain  an  effective  enlisted  force. 

The  primary  constraint  on  the  model  proposed  here  is  that  a  cohort 
of  accessions  should  achieve  a  prescribed  level  of  performance.  This  im¬ 
plies  that  any  set  of  enlistment  standards  must  produce  a  cohort  of  acces¬ 
sions  whose  predicted  performance  is  satisfactory  or  better.  Without  this 
constraint,  or  a  similar  one,  the  aggregate  level  of  performance  could  fall 
below  the  required  level  and  the  enlistment  standards  could  have  dire  con¬ 
sequences. 

Determining  this  prescribed  level  of  performance  is  a  difficult  task  but 
one  that  is  implicitly  done  in  each  of  the  services.  All  services  have  devel¬ 
oped  manning  documents  that  detail  the  force  structure.  The  structure  is 
defined  in  terms  of  the  number  of  people  authorized  at  each  skill  level  or 
rank.  Through  years  of  experience,  policy  makers  have  decided  in  general 
terms  what  level  of  performance  to  expect  from  people  at  various  ranks. 
The  proposed  Marine  Corps  procedure  attempts  to  formalize  the  decision¬ 
making  process  by  having  job  experts  rate  the  level  of  performance  expected 
from  people  at  each  rank  in  an  occupational  specialty. 


The  ratings  of  expected  performance  could  be  in  terms  of  the  percent¬ 
age  of  job  requirements  that  individuals  at  each  rank  should  be  able  to  per¬ 
form.  Thus  the  ratings  of  expected  performance  could  be  directly  compared 
to  the  level  of  predicted  performance  (from  the  regression  analysis).  The 
standards  of  expected  performance  would  be  judged  at  two  levels:  desirable 
(e.g.,  applied  during  good  conditions  when  an  adequate  supply  of  qualified 
people  is  available  for  promotion)  and  minimum  (e.g.,  applied  when  people 
must  be  promoted  before  they  are  fully  qualified).  These  judgments,  if  they 
can  be  made  reliably,  tc7'  't  with  the  authorized  grade  structure,  would 
determine  the  distrib’  jrformance  required  in  an  occupational  spe¬ 
cialty.  The  predic'  ^rmance  of  a  cohort  of  accessions  should  satisfy 

the  expected  perfr.mance  required  in  that  specialty. 


Estimating  Costs 

Estimating  costs  associated  with  operating  a  force  is  an  onerous  process, 
particularly  as  differential  cost  estimates  are  required  for  various  accession 
strata  to  obtain  more  precise  evaluations  of  the  cost-effectiveness  of  enlist¬ 
ment  alternatives.  It  is  often  the  case  that  data  are  not  available  nor  in 
proper  form  for  the  determination  of  costs  associated  with  particular  as¬ 
pects  of  operating  a  force.  This  section  does  not  focus  on  how  such  cost 
estimates  can  be  obtained,  but  rather  details  cost  factors  that  might  po¬ 
tentially  be  considered  as  important  for  inclusion  in  the  cost-performance 
model. 

•  Recruiting.  It  is  generally  the  premise  that  higher-ability  personnel 
are  more  costly  to  recruit  than  their  lower-ability  counterparts.  This 
is  reflected  in  allocation  of  the  recruiters’  time,  effort,  and  travel  ex¬ 
penses  in  seeking  to  enlist  high  school  graduates.  Also,  additional 
expenses  are  incurred  by  the  services  as  enlistment  bonuses  or  edu¬ 
cational  programs  are  used  as  incentives  to  lure  the  more  qualified 
personnel. 

•  Training.  While  it  may  be  argued  that  all  personnel  receive  the  same 
basic  and  even  advanced  training  (in  this  case,  training  would  be  a 


fixed  cost),  it  may  be  true  that  some  personnel  groups  are  trained  to 
differing  levels  of  proficiency.  This  may  be  reflected  in  the  length  of 
training  necessary  to  achieve  a  mastery  level  of  the  subject  matter 
or,  if  training  time  is  fixed,  differential  training  pass  rates.  The  costs 
of  training  are  magnified  even  more  for  certain  personnel  groups  if 
individuals  who  fail  are  recycled  back  through  the  training  program 
or  assigned  to  another  specialty. 

Force  Maintenance.  In  many  instances,  force  maintenance  costs  can 
be  considered  fixed  as  they  apply  to  all  personnel  regardless  of  their 
ability  or  educational  level.  However,  to  the  extent  that  such  factors 
as  pay,  bonuses,  leave,  accomodations,  or  other  such  amenities  are 
differentially  allocated  to  personnel,  these  costs  should  be  explicitly 
associated  with  the  particular  personnel  group. 

Attrition.  Given  that  individuals  who  leave  the  service  must  be  re¬ 
placed,  there  are  costs  associated  with  recruiting,  training,  and  main¬ 
taining  a  replacement  to  a  performance  level  on  par  with  the  individ¬ 
ual  who  left.  To  the  degree  that  some  groups  of  individuals  have  a 
greater  propensity  to  drop  out  than  others,  variable  costs  associated 
with  these  groups  will  be  experienced  by  the  services. 

Discipline.  Some  personnel  tend  to  be  reprimanded  for  misconduct 
more  frequently  than  others.  To  the  extent  that  the  service  must 
maintain  facilities  and  manpower  to  address  this  problem,  experience 
personnel  inadequacies  or  losses  in  certain  areas,  realize  substandard 
job  performance,  or  repair  any  damage  resulting  from  the  misconduct, 
these  types  of  costs  accumulate  and  should  possibly  enter  into  the  cost 
formulations  for  evaluation  of  alternative  standards. 

Reenlistment.  Not  all  persons  qualify  for  reenlistment.  Indirect  costs 
are  associated  with  those  personnel  who  do  not  qualify,  because  the 
service  has  a  great  deal  of  training  and  on-the-job  experience  invested 
in  that  individual.  If  certain  groups  of  personnel  are  more  likely  to 
qualify  than  others,  then  this  could  be  considered  as  a  variable  in  the 
estimation  of  total  costs. 


The  determination  of  costs  associated  with  these  factors  is  an  ambitious 
but  required  endeavor.  Certainly,  there  are  other  cost  factors  that  can  be 
included,  but  given  that  the  cost  estimates  are  probably  correlated  across 
personnel  groups,  the  inclusion  of  additional  factors  will  only  magnify  the 
differential  costs.  Therefore,  the  careful  estimation  of  some  subset  of  the 
above  costs,  as  opposed  to  the  incomplete  estimation  of  all  factors,  will 
at  least  provide  a  conservative  or  lower  bound  estimate.  The  first  three 
factors  are  probably  the  most  pertinent  to  the  issue  of  setting  enlistment 
standards  (attrition  and  discipline  can  possibly  be  subsumed  under  force 
maintenance) . 

Evaluating  Alternative  Enlistment  Standards 

The  final  step  of  the  linkage  process  is  to  evaluate  alternative  enlistment 
standards  in  terms  of  their  cost-effectiveness.  That  is,  which  cutoff  score  on 
the  aptitude  composite  will  achieve  the  prescribed  level  of  expected  perfor¬ 
mance  for  the  least  cost?  Conceptually,  this  is  the  easiest  stage  because  the 
costs  are  simply  computed  for  differing  standards  with  the  resulting  quality 
mix  of  accessions  being  determined  by  the  previous  relationship  established 
between  job  performance  and  accession  attributes  (step  2).  Operationally, 
this  step  is  more  difficult.  However,  the  Rand  Corporation  has  developed 
a  prototype  model  that  synthesizes  the  job  performance  criterion  data,  the 
cost  estimates,  and  the  determined  job  performance  -  accession  attribute 
relationship  [2,3]. 

The  Rand  model  actually  defines  two  composite  performance  crite¬ 
rion  measures:  one  that  reflects  an  individual’s  ability  to  remain  in  the 
service  (retained  man-months)  and  another  that  extends  this  retention 
measure  by  including  an  assessment  of  on-the-job  performance  (qualified 
man-months).  The  first  Rand  study  [2]  constrained  the  number  of  re¬ 
tained  man-months  while  optimizing  qualified  man-months  at  minimum 
cost.  The  results  showed  that  force  end-strength  figures  were  reasonably 
well  maintained.  However,  it  should  be  noted  that  retained  man-month  is 
an  aggregate  number  that  is  not  dependent  on  the  distribution  of  predicted 
performance.  Thus,  it  is  possible  that  cost-effective  enlistment  standards 
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constrained  by  retained  man-months  would  possibly  not  result  in  a  distri¬ 
bution  of  predicted  performance  needed  for  sustaining  the  career  enlisted 
force.  Conversely,  the  number  of  qualified  man-months  could  have  been 
constrained  instead  and  total  costs  minimized.  In  this  case,  force  size  can 
vary,  which  may  not  be  appropriate  either. 

In  addition,  the  qualified  man-month  performance  measure,  as  applied 
in  the  Rand  papers  [2,3],  is  essentially  a  dichotomous  variable  based  on  an 
individual’s  ability  to  pass  the  Army  Skills  Qualification  Test  (SQT).  The 
model  treats  individuals  who  just  barely  pass  the  SQT  as  comparable  to 
those  who  perform  extremely  well  on  the  test.  In  this  manner,  individ¬ 
ual  differences  are  potentially  ignored.  Accordingly,  the  model  will  favor 
recruiting  individuals  who  score  just  above  the  cutoff  score  on  the  SQT  be¬ 
cause  they  are  generally  less  expensive  to  recruit.  Thus,  it  is  recommended 
that  a  continuous  performance  measure  (like  the  predicted  performance 
variable  discussed  earlier)  be  incorporated  in  the  Rand  model.  This  should 
be  a  rather  simple  substitution  in  that  a  mean  performance  score  (e.g., 
AFQT  category  I  personnel  average  94  percent  correct  on  the  SQT)  could 
replace  the  predicted  SQT  pass  rates  (e.g.,  96  percent  of  AFQT  category  I 
personnel  pass  the  SQT). 

Sur  lary  Remarks 

The  issue  of  linking  job  performance  measures  to  enlistment  standards 
has  not  received  the  attention  that  it  deserves  by  the  Job  Performance  Mea¬ 
surement  Working  Group.  The  primary  concern  of  this  working  group  and 
the  services  was  to  develop  the  necessary  technology  and  gain  experience 
in  constructing  hands-on  job  performance  tests.  Now  that  each  of  the  ser¬ 
vices  have  gained  this  experience  and  have  collected  performance  data,  the 
linkage  topic  will  certainly  come  to  the  forefront.  However,  except  for  the 
Rand  studies,  there  is  little  experience  to  build  upon  in  the  area  of  relating 
hands-on  performance  criterion  to  tests  of  cognitive  functioning.  Thus,  the 
challenge  faced  by  the  services  is  to  develop  a  better  understanding  of  what 
is  involved  in  the  linkage  process.  This  paper  has  sought  to  address  this 
concern  by  briefly  outlining  the  steps  involved  in  the  process.  Certainly. 
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these  steps  do  not  encompass  the  entire  linkage  process,  but  at  least  they 
may  provide  a  starting  point  for  discussion  among  the  services. 


In  addition  to  gaining  a  firmer  technical  understanding  of  relating  per¬ 
formance  measures  to  ASVAB  scores,  it  is  imperative  that  each  service 
recognize  the  importance  of  involving  their  policy  makers  from  the  very  be¬ 
ginning.  The  determination  of  enlistment  standards  is  an  extremely  policy- 
oriented  decision.  From  the  defining  of  job  requirements  and  the  selection 
of  test  content  to  the  determination  of  expected  performance  required  by 
the  service,  important  decision  must  be  made  by  the  service  experts  and 
policy  representatives.  To  the  extent  that  policy  makers  are  involved  in  the 
entire  linkage  process,  they  will  make  better  and  more  informed  decisions. 
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